Learning Q-Function Approximations for Hybrid Control Problems

The main challenge in controlling hybrid systems arises from having to consider an exponential number of sequences future modes make good long-term decisions. Model predictive control (MPC) computes a action through finite-horizon optimisation problem. A key ingredient this problem is terminal cost, account for the system's evolution beyond chosen horizon. cost can reduce horizon length required and often tuned empirically by observing performance. We build on idea using N-step Q-functions ( Q (N) ) MPC objective avoid choose cost. present formulation incorporating system dynamics constraints approximate optimal -function algorithms train approximation parameters exploration state space. test policy derived trained approximations two benchmark problems simulations observe that our are able learn -approximations with dimensions practical relevance based relatively small data-set. compare controller's performance against Hybrid terms computation time closed-loop costs.